This invention relates to networked client/server systems and to methods of streaming and rendering multimedia content in such systems.
Multimedia streamingxe2x80x94the continuous delivery of synchronized media data like video, audio, text, and animationxe2x80x94is a critical link in the digital multimedia revolution. Today, streaming media is primarily about video and audio, but a richer, broader digital media era is emerging with a profound and growing impact on the Internet and digital broadcasting.
Synchronized media means multiple media objects that share a common timeline. Video and audio are examples of synchronized mediaxe2x80x94each is a separate data stream with its own data structure, but the two data streams are played back in synchronization with each other. Virtually any media type can have a timeline. For example, an image object can change like an animated .gif file: text can change and move, and animation and digital effects happen over time. This concept of synchronizing multiple media types is gaining greater meaning and currency with the emergence of more sophisticated media composition frameworks implied by MPEG-4, Dynamic HTML, and other media playback environments.
The term xe2x80x9cstreamingxe2x80x9d is used to indicate that the data representing the various media types is provided over a network to a client computer on a real-time, as-needed basis, rather than being pre-delivered in its entirety before playback. Thus, the client computer renders streaming data as it is received from a network server, rather than waiting for an entire xe2x80x9cfilexe2x80x9d to be delivered.
The widespread availability of streaming multimedia enables a variety of informational content that was not previously available over the Internet or other computer networks. Live content is one significant example of such content. Using streaming multimedia, audio, video, or audio/visual coverage of noteworthy events can be broadcast over the Internet as the events unfold. Similarly, television and radio stations can transmit their live content over the Internet.
In comparison to text-based or paper-based presentations, streaming multimedia presentations are very effective in certain situations. Audio/visual presentations, for example, are able to capture and convey many subtle factors that are not perceivable from paper-based documents. Even when the content is a spoken presentation, an audio/visual recording captures gestures, facial expressions, and various speech nuances that cannot be discerned from text or even from still photographs.
Although streaming multimedia content compares favorably with textual content in most regards, one disadvantage is that it requires significant time for viewing. It cannot be xe2x80x9cskimmedxe2x80x9d like textual content. Thus, information consumers are forced to choose between the efficiency of the written word and the richness of the multimedia experience.
The invention described below addresses this disadvantage of prior art streaming multimedia content, allowing more efficient multimedia perusal of streaming multimedia presentations than has previously been possible.
The invention utilizes time-scale modification so that a user can vary the speed of streaming content without destroying its intelligibility. In accordance with the invention, a user selects multimedia content from a menu presented at a network client computer. In addition, the user selects a speed factor, indicating the speed at which the multimedia should be rendered relative to its default speed.
In response to these selections by the user, the network client contacts a network server and initiates a composite media stream from the server to the client. As it receives the media stream, the client processes the individual media streams of the composite stream, altering their timelines in accordance with the user""s speed designation. Time-scale modification techniques are used to alter audio streams to maintain their original pitch.
In alternative embodiments, timelines are altered at the server, before streaming to the client. To accomplish this, the server either alters the timelines dynamically, as specific content is requested, or selects from a number of prestored media streams having pre-altered timelines.
The invention includes methods of adapting to limited bandwidth situations by composing or selecting composite streams having differing degrees of quality, and/or by composing or selecting streams with timelines that are altered to closely correspond with whatever speed factor has been chosen. In one embodiment of the invention, certain media streams, such as audio streams, take precedence over other streams such as video streams. In this embodiment of the invention, the audio stream is sent with an unaltered timeline, at a rate sufficient to satisfy the consumption requirements of the client, given the current speed factor. The video is then degraded in quality to reduce its bandwidth, so that it can be streamed in whatever bandwidth is not require by the audio.